Instruction level runtime behavioral analysis can detect evasive malware attacks involving return oriented programming (ROP) attacks, providing high detection accuracy and a very low false positive rate. However, binary translation (BT) or instrumentation techniques used in such analysis incur overhead in dynamically instrumenting code for enabling runtime instruction level execution monitoring and detecting execution anomalies. In addition, other issues can also impact performance. For example, binary translation relies on code injection or just in time (JIT) compilation techniques to re-generate original code with security checks, which may become a potential attack surface from malware if not properly protected. In addition, dynamic code injection may not be possible when disallowed by a security policy for a given process.
Binary translation, which dynamically re-generates an instruction stream with in-lined security checks, can be complex, and may also compel runtime support for allocating and garbage collecting a translation cache (storage for re-generated code) during runtime, and may also require complex solutions for detecting self and cross-modifying code conditions to invalidate the translated code for correctness. In addition, binary translation techniques insert monitoring software inside the application itself, which may not be desirable when seeking complete secure isolation of security monitoring software.